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Remarks 

The drawing figures have been corrected to show the dashed lines more clearly. However, 
the Examiner's objection with regard to reference numerals 24, 25, and 26 is respectfully 
traversed since these appear in the description in paragraph 42 at the top of page 8. 

Paragraph 57 has been amended to correctly identify the operator's head 105. 

An information disclosure statement is being submitted herewith. 

Claim 1 has been amended to better define the invention and overcome the informalities 
noted by the Examiner. Similar amendments have been made to claim 18. 

Claims 7 to 9 have been re-worded in method format. Claim 10, which is more properly 
an apparatus claim, has been made dependent on claim 18. 

With regard to claim 17, it is respectfully submitted that this is clear as amended. A 
captured image may have some degree of perspective distortion, as would be recognized 
by one skilled in the art. Claim 17 merely calls for the perspective distortion present to be 
reduced using the range data. Thus, the base level is the distortion present in the raw 
captured data. 

Claim 22 now specifies that the de-emphasis is relative to current image data (see 
paragraph 690. 

A fundamental difference between the present invention is that in the present invention 
the images are sent to the processor for immediate display , either of the direct image if 
there is no offset between the current field of view of the HMD and the camera's field of 
view at the time the image was captured, and of a transformed image corresponding to the 
overlapping portion of the two fields of view if an offset is present. In Hirose, a virtual 
omnimax image is built up, and the HMD derives its data primarily from the virtual 
omnimax image, with a provision (see 2.3) to preferentially refresh the image in the 
current direction of the HMD. 

The significance of this difference is that Hirose, while concerned with the same problem 
of minimizing latency, is only applicable to static systems. His system is not applicable 
situations where the camera is in motion, for example as might be found on the space 
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shuttle, since the surrounding scene is constantly changing. Therefore, the imagery in his 
virtual omnimax would never correspond to the actual position of the camera. 

Claims 1 and 18 are distinguished in many respects from the teachings of Hirose. For 
example, Hirose does not teach tagging the images with the CMD data, which is essential 
in the invention for the processor to be able to determine whether there is an offset 
between the HMD position and the camera position associated with the currently 
displayed data. Hirose does not teach sending a sequence of video images to the processor 
for display under the circumstances claimed, neither does he teach comparing the HMD 
position data with the CMD data for each captured image to determine whether an offset 
exists. 

Unlike Hirose, which is not truly real time because it relies on continuously scanning a 
surrounding space to build up a virtual omnimax, the present invention is a real time 
system wherein the operator can feel as if he is viewing a scene at a remote location, even 
if the camera is moving . It turns out that a short delay in the equipment responding to 
head movement is very nauseating for the operator. By creating an immediate response, 
the invention solves this problem, and unlike Hirose, solves it in a way that works when 
the camera is in motion. 

In section 2.3, Hirose briefly discusses "transmission of real-time information". He 
recognizes that limitations arise in the ability of the system to provide real-time 
information "due to capacity limitations of the system". His data always comes initially 
from the virtual dome, which is stored in computer memory. His solution is to 
preferentially refresh the images in the direction in which the user is looking. He says that 

"when a user changes his head orientation, he will initially see previously 
obtained images for a few instances. However, if the user continues looking in the 
same direction for a noticeable period of time, images from the remote camera 
will refresh the old ones and portray more recent images of the remote location. 
Using this approach, the user can essentially see live video images without losing 
his sense of orientation. 

The operative word here is essentially. The Hirose system will never be truly real time 
because of the need to refresh the images stored in the virtual dome. He suggests giving 
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priority to refreshing images in the dome in the direction in which the user is looking, but 
in his system he still needs to scan the whole space to refresh the dome. "Currently, a 
system has been implemented in which entire images of the surrounding space are first 
captured by the rotating camera head and sent to the workstation." Also, Hirose states that 
"the camera head unit is located in the remote place and continually scans its surrounding 
space." See passage bridging pages 125 and 126. 

By contrast, the invention the camera tracks head movement. Hirose teaches away from 
this approach since he states (para 2. 1) that "The key principle of the Virtual Dome was 
to separate the camera head movements and the user's head movements". 

If the proposed modification or combination of the prior art would change the 
principle of operation of the prior art invention being modified, then the teachings 
of the references are not sufficient to render the claims prima facie obvious. In re 
Ratti, 270 F.2d 810, 123 USPQ 349 (CCPA 1959) 

Since the key principle of operation of Hirose is to separate the camera head movements 
and the user's head movements, whereas in the invention the opposite is the case, namely 
the camera tracks the user's head movements, it cannot be obvious to modify Hirose to 
arrive at the invention. 

In commenting on claim 10, which claims the mechanism for moving the camera to 
follow the head mounted display, the Examiner referred to section 1 . With respect the 
Examiner cannot mix apples and oranges. Section 1 refers to a conventional prior art 
system, wherein movement of the head mounted display controls movement of the 
camera. Such a prior art system suffers from the disorientation problem the invention 
solves because of the delay in the camera tracking the operator's head movement. There 
is no disclosure of such a system in combination with any means to compensate for 
motion delay. Hirose then goes on in section 2 to say that his principle is NOT to have the 
camera track the HMD movement but to separate the two. He then goes on to discus how 
to achieve this as discussed above. Hirose cannot therefore be regarded as a teaching of a 
HMD tracking system in combination with some system for storing or transforming 
images because Hirose expressly teaches that unlike the prior art his system does not 
track head movement. There is no logical way that Hirose's dome could be used with a 
head tracking system. Such a combination is directly contrary to his teaching. 
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The prior art has to be read for what it teaches as a whole. It is not permissible just to 
extract portions of a document and piece them together in a way that is directly contrary 
to the teaching of the document. To do that is to make a hindsight reconstruction of the 
invention using the applicant's own teachings. 

prior art reference must be considered in its entirety, i.e., as a whole, including 
portions that would lead away from the claimed invention. W.L. Gore & 
Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), 
cert, denied, 469 U.S. 851 (1984) 

Moreover, there is no teaching of associating the actual images with the CMD data as 
claimed and comparing the CMD data with the current HDM date to determine whether 
an image transformation is necessary. 

Claims 1 and 18 are therefore believed patentable, as are the remaining claims which are 
dependent thereon. 

Allowance and reconsideration are therefore earnestly solicited. 
Respectfully submitted, 
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